⚗️Use ZSTD compression standard when building images instead of default gzip #8535
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #8535 +/- ##
==========================================
+ Coverage 87.23% 87.55% +0.31%
==========================================
Files 1935 2012 +77
Lines 76477 79025 +2548
Branches 1368 1368
==========================================
+ Hits 66717 69192 +2475
- Misses 9354 9427 +73
Partials 406 406
Continue to review full report in Codecov by Sentry.
🚀 New features to boost your workflow:
|
🧪 CI InsightsHere's what we observed from your CI run for d6c193c. ✅ Passed Jobs With Interesting Signals
|
sanderegg
force-pushed
the
docker-use-zstd
branch
from
23b9cf6 to
3842f3f
Compare
sanderegg
requested review from
YuryHrytsuk,
mrnicegyu11 and
pcrespov
as code owners
There was a problem hiding this comment.
Pull Request Overview
This PR switches Docker image compression from the default gzip to ZSTD with compression level 3 when pushing images to the registry. The change aims to improve compression efficiency during the build and push process.
Key Changes:
- Modified the Makefile build process to use ZSTD compression with level 3 for registry pushes
- Updated local build output to use OCI format with ZSTD compression level 9
- Removed extraneous blank line in docker-compose.yml
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| Makefile | Updates build output configuration to use ZSTD compression (level 9 for local, level 3 for registry) and switches to OCI format |
| services/docker-compose.yml | Removes blank line between comment and environment variable definition |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
Thanks! What is the influence of this change? Shall PS I would also explain "why" in the PR's description |
@pcrespov @YuryHrytsuk @mrnicegyu11 some additional infos were added in the PR description. I hope this makes things clearer. |
4 participants
Why this PR
In a continuous effort to make the osparc leaner, faster to build and/or faster to deploy this is a change in how we use docker buildX (which is already a few years old) to build our containers.
To recall, when we use
docker builda docker image is built which is made of "layers". Each of these layers are actually some docker instructions that were executed and compressed and stored. The original, and default compression algorithm uses gzip, which is >20 years old. gzip can be compressed only single-threaded and decompressed multi-threaded if the library pigz is available (which we have on our deployments).Somewhere in the 2000's Meta developed another compression algorithm called zstd, which allows for multi-threaded compression and decompression. Also from benchmarks around (AWS Fargate) it seems that the performance for decompressing can be up to 50% than the usual gzip. This is where it is interesting for autoscaled machines, especially for cold starts, but also to prepare warm buffers as they would need to be up for a smaller amount of time. Also another good thing is that the size of the images is reduced vs traditional gzip, which means less data to transfer (docker source)
What do these changes do?
make build push=truenow uses ZSTD compression standard and compression level = 3docker pushwhich always returns to old gzipRelated issue/s
How to test
Dev-ops